Skip to main content

Reply to this post | Go Back
View Post [edit]

Poster: brewster Date: Nov 27, 2002 3:24am
Forum: movies Subject: Techy volunteer needed: setup linux tools for conversion

The archive could really use the help for converting mpeg-2 files to vcd files (mpeg-1) and mpeg-4 files. There are tools around to do the former, and maybe the latter on linux. If you are up for this, pls reply to this post.

thank you!

-brewster

Reply to this post
Reply [edit]

Poster: Doug Merritt Date: Nov 27, 2002 9:12am
Forum: movies Subject: Re: Techy volunteer needed: setup linux tools for conversion

Coincidentally, I just started looking into this
for reasons of personal interest,
so I hereby volunteer to do some initial
investigation, and possibly more than that.

I understand the technical background on the
subject, but I haven't fiddled much with
freely-available mpeg tools on Linux yet,
so I'm not sure how much total work will be
involved, and hence can't give a firm commitment
to a final deliverable (especially if I become
employed soon :-)

Is there more on your requirements list than
what you already said, or is it simply a
desire to provide more formats?

For instance, is use of commercial software
desirable or undesirable? (I favor open source,
and preferably free, when it works to do so,
but it doesn't always work that way.)

Do you want quality guarantees or just best
effort? If a package introduces certain
artifacts by re-sampling audio or images
to a different frame-rate/sampling rate
(a known can of worms), is that acceptable
or unacceptable?

Is it desirable to retain the exact original
bits whenever possible, e.g. so that round-trip
conversion is at least theoretically approximately
possible, or is that unimportant, or is
degradation (e.g. for sake of speed) actually
desirable, or is audio/image "improvement"
desirable? (E.g. from scan line interpolation,
motion tracking and interpolation, DCT artifact
pre-compensation, etc etc etc)

I have my own opinions on all of this. If it
were me, I'd at least initially go for least
impact on the original data, trying neither to
degrade it nor to enhance it, but not worrying
too much either way for the first release, since
the converted format is not going to be a
precious archival version in itself.

In a second release it would be interesting to
aim at enhancement, for better user experience --
although the counter-argument is that such
should go into user viewing software. But what
if it can't be done in real time on user
equipment in the near term?

Counter-counter-argument: then it would soak
up too much server cpu. (The argument doesn't
end there, either, of course.)

I'm not sure of the right answer, but I do
think that e.g. early Edison Archive films, for
instance, are in fact very badly in need of all
the enhancement they can get, for the sake
of the user experience, but not in the archived
bits, which need to be preserved as exactly
as possible for posterity, future archaeology,
future better algorithms, etc.

Sorry to be long winded. It's a pet subject of
mine of long standing. :-)
Doug

Reply to this post
Reply [edit]

Poster: brewster Date: Nov 27, 2002 11:42am
Forum: movies Subject: Re: Re: Techy volunteer needed: setup linux tools for conversion


thank you for the offer. you are on!

below at the >> are some answers to your questions.

Poster: Doug Merritt Date: November 27, 2002 05:12:56pm
Forum: movies Subject: Re: Techy volunteer needed: setup linux tools for conversion
Coincidentally, I just started looking into this
for reasons of personal interest,
so I hereby volunteer to do some initial
investigation, and possibly more than that.

I understand the technical background on the
subject, but I haven't fiddled much with
freely-available mpeg tools on Linux yet,
so I'm not sure how much total work will be
involved, and hence can't give a firm commitment
to a final deliverable (especially if I become
employed soon :-)

>> sounds good.

Is there more on your requirements list than
what you already said, or is it simply a
desire to provide more formats?

>> we want to provide VCD and Mpeg-4 so that people
>>will lower bandwidth connections can see the
>>movies without the heavy weight download of
>>an mpeg-2.

For instance, is use of commercial software
desirable or undesirable? (I favor open source,
and preferably free, when it works to do so,
but it doesn't always work that way.)

>> we like to run on a cluster, so command line
>>oriented, and not tied to one cpu are
>>highly desirable.

Do you want quality guarantees or just best
effort? If a package introduces certain
artifacts by re-sampling audio or images
to a different frame-rate/sampling rate
(a known can of worms), is that acceptable
or unacceptable?

>> degradation is acceptable.


Is it desirable to retain the exact original
bits whenever possible, e.g. so that round-trip
conversion is at least theoretically approximately
possible, or is that unimportant, or is
degradation (e.g. for sake of speed) actually
desirable, or is audio/image "improvement"
desirable? (E.g. from scan line interpolation,
motion tracking and interpolation, DCT artifact
pre-compensation, etc etc etc)

>> we should do what we can as easily as we can.
>> the mpeg-2 are our current reference standards
>> so people can go back to those.


I have my own opinions on all of this. If it
were me, I'd at least initially go for least
impact on the original data, trying neither to
degrade it nor to enhance it, but not worrying
too much either way for the first release, since
the converted format is not going to be a
precious archival version in itself.

In a second release it would be interesting to
aim at enhancement, for better user experience --
although the counter-argument is that such
should go into user viewing software. But what
if it can't be done in real time on user
equipment in the near term?

>> viewing should be realtime, encoding can be
>> way slow. we have lots and lots of machines
>> we can throw at this problem.

Counter-counter-argument: then it would soak
up too much server cpu. (The argument doesn't
end there, either, of course.)

I'm not sure of the right answer, but I do
think that e.g. early Edison Archive films, for
instance, are in fact very badly in need of all
the enhancement they can get, for the sake
of the user experience, but not in the archived
bits, which need to be preserved as exactly
as possible for posterity, future archaeology,
future better algorithms, etc.

Sorry to be long winded. It's a pet subject of
mine of long standing. :-)
Doug

>> pls send me an email at brewster at archive org
>> so I can set up an account for you.
>> -brewster

------------------------------------------------------------------------

Reply to this post
Reply [edit]

Poster: oliver Date: Nov 27, 2002 9:00pm
Forum: movies Subject: Re: Techy volunteer needed: setup linux tools for conversion

mplayer could handle that:
http://www.mplayerhq.hu/DOCS/#command_line

Reply to this post
Reply [edit]

Poster: Doug Merritt Date: Dec 1, 2002 7:57am
Forum: movies Subject: Re: Techy volunteer needed: setup linux tools for conversion

Oliver wrote:
> mplayer could handle that

Thanks for the tip.

Reply to this post
Reply [edit]

Poster: oliver Date: Dec 1, 2002 8:12am
Forum: movies Subject: Re: Techy volunteer needed: setup linux tools for conversion

if things need to be scripted, consider
"mencoder" which comes with mplayer. Examples:
http://www.mplayerhq.hu/DOCS/encoding.html

Reply to this post
Reply [edit]

Poster: akb Date: Dec 2, 2002 3:55am
Forum: movies Subject: Re: Techy volunteer needed: setup linux tools for conversion

Mpeg4 is a little sketchy with mplayer/mencoder. You might take a look at http://mpeg4ip.sf.net, its a GPL implementation sponsored by Cisco.

Reply to this post
Reply [edit]

Poster: Doug Merritt Date: Dec 10, 2002 5:13am
Forum: movies Subject: Progress on: Techy volunteer needed: setup linux tools for conversion

Thanks for the mpeg4ip tip.

Mplayer has not worked out (and I was only trying
to compile the command-line version).

It sounds very promising, since it tries to
support every video format, however the
authors have an over-emphasis on state of the
art C compilers (in order to get every bit of
performance possible, I suppose). It would
not compile -- actually configure refused to
continue -- with my system's egcs 2.91.66
(two years old, I think). Using such an
obsolete compiler is a fatal error to MPlayer's
configure. Sigh.

The instructions told me I could install
just the very latest gcc-core 3.1.1, which didn't
compile, and when I fixed that by hand, it turned out
that the very latest binutils was also needed,
which also didn't compile but was not trivially
fixable.

I could spend a bunch of time upgrading my entire
2-year-old Linux distribution, however as I understand
it, Brewster is interested in tools that are
as portable as possible, and MPlayer is
demonstrating that they are interested in the
opposite...nor am I completely confident that
it would compile even *then*...so another tool
is probably in order.

I did successfully use vcdimager to convert,
however it produces an entire VCD file system,
not just a new mpeg file, and I don't see an
option to do otherwise. (I *think* I was successful;
my non-computer DVD player refuses to handle
CD-Rs and CD-RWs.)

Reply to this post
Reply [edit]

Poster: akb Date: Dec 10, 2002 6:24am
Forum: movies Subject: Progress on: Techy volunteer needed: setup linux tools for conversion

Hmm, actually Redhat caused quite a bit of controversy with their bundling of gcc2.96 which had known problems. There are a bunch of projects which it breaks, the linux kernel being one of them. I've had no problems with gcc3.2 myself and just compiled mplayer with it the other day pain free. It would still be my choice for use on this type of project.

Reply to this post
Reply [edit]

Poster: Doug Merritt Date: Dec 10, 2002 7:37am
Forum: movies Subject: Progress on: Techy volunteer needed: setup linux tools for conversion

It's SuSE, not Redhat, but in any case, are you
trying to tell me that egcs 2.91.66 is somehow
equivalent to gcc 2.96? Mplayer instructions
certainly make it clear that 2.96 shouldn't be
used, but I didn't see anything about egcs.

Reply to this post
Reply [edit]

Poster: akb Date: Dec 10, 2002 7:42am
Forum: movies Subject: Progress on: Techy volunteer needed: setup linux tools for conversion

I'm not perfect on my gcc lineage but the whole 2.9x and egcs merge was rocky. I would suggest 3.2, I haven't had any problems compiling anything with it and it seems to have the blessing of the free software community generally.

Reply to this post
Reply [edit]

Poster: akb Date: Dec 11, 2002 2:11am
Forum: movies Subject: Re: Progress on: Techy volunteer needed: setup linux tools for conversion

After hearing about your frustrations, I spent a little time on this last night and fiddled around with a few different tools. My general goal was to get as close to the already converted prelinger vcd and divx movies, though I'm not sure if that's exactly what Brewster wanted.

I settled on ffmpeg, mplayer uses its libraries for a good bit of its functionality but its frontend didn't let me do what I wanted with the mpeg conversions. Mplayer would be nice if you want to easily try other mpeg4 codec implementations, like Xvid or the binary ones for Windows.

Once you play around with the options to get what you want its pretty easy to script to convert from mpeg2 automatically. One thing that would take a bit of work would be to test if it handled mpeg2's in various aspect ratios etc.

Here's what I settled on.

ffmpeg -i preview1.mpg -s 352x280 -b 1150.5 -ab 224 preview.mpg1

ffmpeg -i 10816a.mpg -s 320x240 -b 452.5 -vcodec mpeg4 -acodec mp3 -ab 64 10816a.avi

Reply to this post
Reply [edit]

Poster: Doug Merritt Date: Dec 10, 2002 5:13am
Forum: movies Subject: Progress on: Techy volunteer needed: setup linux tools for conversion

Thanks for the mpeg4ip tip.

Mplayer has not worked out (and I was only trying
to compile the command-line version).

It sounds very promising, since it tries to
support every video format, however the
authors have an over-emphasis on state of the
art C compilers (in order to get every bit of
performance possible, I suppose). It would
not compile -- actually configure refused to
continue -- with my system's egcs 2.91.66
(two years old, I think). Using such an
obsolete compiler is a fatal error to MPlayer's
configure. Sigh.

The instructions told me I could install
just the very latest gcc-core 3.1.1, which didn't
compile, and when I fixed that by hand, it turned out
that the very latest binutils was also needed,
which also didn't compile but was not trivially
fixable.

I could spend a bunch of time upgrading my entire
2-year-old Linux distribution, however as I understand
it, Brewster is interested in tools that are
as portable as possible, and MPlayer is
demonstrating that they are interested in the
opposite...nor am I completely confident that
it would compile even *then*...so another tool
is probably in order.

I did successfully use vcdimager to convert,
however it produces an entire VCD file system,
not just a new mpeg file, and I don't see an
option to do otherwise. (I *think* I was successful;
my non-computer DVD player refuses to handle
CD-Rs and CD-RWs.)